This section provides background information related to the present disclosure which is not necessarily prior art. This section provides a general summary of the disclosure, and is not a comprehensive disclosure of its full scope or all of its features.
Typically, vision systems monitor a scene using electromagnetic radiation usually from a specific spectral band, such as ultra violet (UV), visible (VIS), near infrared (NIR), short wave infrared (SWIR), and long wave infrared (LWIR) radiation. Together, the NIR and SWIR spectral bands are also referred to as the reflected infrared band (as opposed to the emitted radiation from the thermal infrared band), while the LWIR band is referred to as the thermal infrared band.
Infrared (IR) vision systems are used in numerous civilian and military applications. Some vision systems are designed to observe scenes under extreme low illumination by using light amplification or light intensification technologies, such as the night vision device described in U.S. Pat. No. 4,463,252. Different information about a scene can be conveyed through incoming electromagnetic radiation from the various portions of the spectrum. Indeed, many techniques have been developed by combining multi-spectral images of the scenes of interest. For example, U.S. Pat. No. 5,035,472 describes a device that transmits the signal of an image along two separate paths, one directing the signal towards an IR detector and the other directing the signal towards an image intensifier. Then, the IR and intensified images are combined for displaying the information to the user.
The number of autonomous vehicles in use on public roads and in civilian airspace has been increasing steadily, exposing them to hazardous environmental conditions such as slippery roads and aircraft icing conditions. Therefore, autonomous vehicles will likely be required to have control systems configured to receive information regarding not only the surrounding terrain and the obstacles in their path, but also the conditions of the road and the airspace ahead. In addition, autonomous vehicles will likely need to respond to this information automatically by commanding maneuvers to negotiate terrain, avoid obstacles, and track a particular path in order to avoid potentially hazardous conditions.
Recent accidents involving automobiles employing advanced automation systems have been discussed extensively in the engineering communities and in the media. Accidents frequently occur when vision systems fail due to exposure to intense sunlight (e.g. sun glare). This is a common problem for vision systems that rely on cameras, lidars, or other devices operating in portions of the spectrum strongly affected by sunlight, such as in the visible and near infrared spectral bands.
The principles of the present teachings provide a method for avoiding problems caused by sunlight in devices used for avoiding obstacles, navigating, detecting road conditions (i.e. distinguishing dry roads from wet roads and icy roads, estimating the thickness of water layers), and sensing the atmospheric conditions in the airspace around an aircraft or an autonomous air vehicle (i.e. detecting potentially hazardous icing conditions or volcanic ash ahead). In addition, the present teachings can be used in systems for detecting the concentration of gases leaking from industrial systems or natural atmospheric constituents.
The present teachings provide a method for determining optimum spectral bands for active vision systems used outdoor and a device that employs this method. The method and the device can be used to provide warnings to drivers, to provide information for aircraft, automobile, and autonomous air, and ground or sea vehicles, among other applications.
In some embodiments of the present invention, detectors, detector arrays, or multi-spectral cameras can be used to make the required measurements. A similar system can be used for detecting ice or water unambiguously on aircraft surfaces, manufacturing systems, or any other object of interest. In some embodiments, a system using measurements in a single optimum spectral band, such as a lidar, can be used for obstacle avoidance or navigation.
In some embodiments of the present invention, a road condition monitoring system is provided that is configured to detect water, snow, frost, clear ice, and other types of ices on roads and any other surface of interest. The system is configured to distinguish dry surfaces from those covered by water, snow, frost, and various types of ice even when these substances cover only a fraction of the field of view of the road condition monitoring system.
Water and ice can often be difficult to detect by drivers or current synthetic vision systems. Clear ice is unusually difficult to detect. Aircrafts, cars, trucks, buses, motorcycles, and other vehicles would benefit from systems capable of detecting the presence of ice or water on surfaces, such as roadways, bridges, sidewalks, or even runway and taxiways (i.e. in connection with ground operations of aircrafts or supporting personal and vehicles). The fact that drivers, operators, and synthetic vision systems fail to detect deteriorating road conditions ahead of a vehicle because of sun glare frequently leads to accidents.
Some of the prior art approaches for vision systems designed for navigation and for the detection of ice and water on roads, aircraft surfaces, and in the airspace around them are based on near infrared radiance measurements. However, these prior art techniques are subject to the negative effects of sun glare because they are based on measurements in portions of the spectrum strongly affect by sunlight.
U.S. Patent App. Pub. No. 2008/0129541AI refers to a slippery ice warning system capable of monitoring the road ahead of a vehicle. One or two visible cameras are used to image the same scene at two orthogonal polarizations. When a single camera is used, a polarization beam splitter is used to separate the reflected light into two orthogonal polarizations. The possible (but ambiguous) determination of the existence of slippery ice ahead of the vehicle is detected by measuring the polarization of the reflected light. Moreover, since this system operates in the visible portion of the spectrum it is subject to the negative effects of sun glare.
U.S. Patent App. Pub. No. 2005/0167593A1 refers to a method that uses shifts in the wavelength of the reflectance near 1.4 μm to distinguish water from ice. In this method, liquid water and ice are discriminated from each other by analyzing shifts in the short wavelength edge of the 1.4 μm band reflectance. Detection decisions are based on shifts in wavelengths in a portion of the spectrum strongly affected by sunlight. Unfortunately, systems based on this method are subject to problems caused by sun glare.
A more recent invention described in U.S. Patent App. Pub. No. 20120193477A1 uses effective reflectance defined as the reflectance of a given material at a given wavelength divided by the reflectance of this same material at a wavelength equal to 1.1 μm to determine the measurements bands for distinguishing wet material from that containing ice on its surface. In this technique, the detection signal is the contrast C between the measurements in the first and the second band, where the contrast signal is defined the ratio between the differences in the intensity of radiation in the second and first bands and the sum of the intensities in the two bands. In this technique, detection decisions are based on the contrast signal is spectral bands strongly affected by sunlight. Unfortunately, systems based on this method are also subject to problems caused by sun glare.
U.S. Pat. No. 9,304,081, which is incorporated herein by reference, describes a technique that uses the radiance ratio around a crossover point (γ=Rλ1/Rλ2) to monitor the condition of the road or the airspace ahead of a land or air vehicle. The key feature of this technique is that the detection signal is robust because if depends simply on the ratio of the measurements in two nearby narrow spectral bands. However, minimization of the effects of sun glare was not a concern when the technique was developed.
According to the principles of the present teachings, optimum spectral bands are implemented in a system for monitoring potentially hazardous conditions ahead such as water, snow, and ice on roads or runways. The system disclosed herein overcomes the disadvantages of the prior art because it is immune to the negative effects of sun glare. Active vision systems can be designed for navigation and for the detection of potentially hazardous conditions in the airspace or in the surface ahead of a vehicle. Industrial systems for outdoors use can be configured to map the system's surroundings or to distinguish clean surfaces from those covered by ice, snow, oil, water, or other particular substances of interest.
In some embodiments of the present teachings, a light source (i.e. a pulsed laser), multispectral detectors and/or multispectral camera, a data processor unit, and interfaces with displays, safety systems, and/or autonomous systems are employed to provide an indication of the conditions ahead and respond to it.
In some embodiments, the road condition monitoring system of the present teachings contains only a detector pair with filters, a data processor unit, and interfaces to displays and control systems. In some embodiments, a lidar using a laser with wavelengths falling within one of the optimum spectral bands described in this document can be used.
Further areas of applicability will become apparent from the description provided herein. The description and specific examples provided in this summary are for illustration only; they are not intended to limit the scope of the present disclosure.
Corresponding reference numerals indicate corresponding parts throughout the several views of the drawings.